1 Data

Replication data are available in SQL and Rdata at https://github.com/judgelord/rulemaking

load(here::here("data", "rules_metadata.Rdata"))

# alternatively 
#rules <- dbGetQuery(con, "SELECT * FROM rules")

#FIXME REPLACE WITH date 
d <- rules %>% mutate(year = str_sub(posted_date, 1,4) %>% as.numeric()) %>%
  filter(year > 2004, year < 2021, document_type %in% c("Proposed Rule", "Rule"))

load(here::here("data", "comments_min.Rdata"))

# hand coded 
load(here::here("data", "coalitions_coded.Rdata"))
load(here::here("data", "comments_coded.Rdata"))
# load(here::here("data", "mass_coded.Rdata"))

# common names 
comments_coded %<>% 
  mutate(coalition = coalition_comment,
         agency = str_remove(docket_id, "-.*"))

# common names 
coalitions_coded %<>% 
  mutate(coalition = coalition_comment,
         comments = coalition_comments,
         Comments = Coalition_comments,
         agency = str_remove(docket_id, "-.*"))

# coalitions_coded %<>% mutate(across(whereis.character, str_to_title)

# TODO merge in mass comments that were not hand-coded

These data currently include 212,516 dockets, 134,927 rulemaking dockets from 1,909-02-10 to 2,020-12-17. These dockets received approximately 99,329,768 comments.

This analysis relies of rulemaking dockets from 2005 through 2020. These 44,583 rulemaking dockets received 75,614,762 comments.

1.1 Clustering with text reuse

My theoretical approach requires that I attribute form letter comments to the organizations, campaigns, and broader coalitions that mobilized them. To do so, I identify comments that share text. I find that a 10-word phrase repeated across more than a few comments is always either text copied from the proposed policy or a form letter provided by a campaign. Thus, for the text of each comment, I first remove all 10-word phrases that appear in the proposed rule (including the preamble and call for comments). Then, I identify all comments that share ten-word phrases with 99 or more other comments. Finally, I collapse these form letter comments to one representative sample for hand-coding.

For each comment on a rulemaking docket1, I identify the percent of words it shares with other comments using a 10-word (or “10-gram”) moving window function, looping over each possible pair of texts to identify matches.2 When actors sign onto the same comment, it is clear that they are lobbying together. However, various businesses, advocacy groups, and citizens often comment separately, even when they are aligned. Text-reuse (using the same ten-word phrases) captures this alignment.

Figure 1.1 shows the percent of shared text for a sample of 50 comments on the Consumer Financial Protection Bureau’s 2016 Rule regulating Payday Loans. Comments are arranged by the document identifier assigned by regulations.gov on both axes. The black on the diagonal indicates that each document has a perfect overlap with itself. Black squares off the diagonal indicate additional pairs of identical documents. For example, 100% of the words from Comment 95976 are part of some tengram that also appears in 95977 because the exact same comment was uploaded twice. The cluster of grey tiles indicates a coalition of commenters using some identical text. Comments 91130 through 91156 are all partial or exact matches. All are part of a mass comment campaign by Access Financial. The percent of the identical text is lower than many mass-comment campaigns because these are hand-written comments, but the n-gram method still picks up overlap in the OCRed text in the header and footer. Tengrams that appear in 100 or more comments indicate a mass comment campaign. Some agencies use similar “de-duping” software [CITE] and only provide a representative sample comment. In these cases, my linking method assumes that the example comment is representative, and I link these comments to others based on the text of the sample comment provided.

knitr::include_graphics(here::here("Figs/comment_percent_match_plot-1.png")  )
Percent of Matching Text in a Sample of Public Comments

Figure 1.1: Percent of Matching Text in a Sample of Public Comments

1.2 Hand-coded sample

To estimate the influence of public comments on policy, I code almost all* comments on a random sample of rules, recording the type of organization, the lobbying coalition to which each belongs, the type of coalition (primarily public or private interests), their policy demands, and the extent to which the change between draft and final rule aligned with their demands. This level of alignment between policy asks and policy outcomes is my measure of lobbying success. It does not identify a causal relationship–true policy influence, but it is state of the art with these kinds of observational data (see Yackee and Yackee (2006)).

*On each selected rule, I code all comments submitted as file attachments or emails, but only some comments typed in a text box. I include comments typed in a text box if they share text with other comments (see above). This includes nearly all comments on most rules, excluding entirely unique text-box content, which is marginal both qualitatively and quantitatively. For comments sharing text, I code one sample document for all versions of the form letter.

My approach to measuring lobbying success starts with policy demands raised in comments. I code the general regulatory/deregulatory direction of the policy change, but the dimensions of conflict on which I judge lobbying success are those identified by commenters. They do not emerge from a reading of the policy or not any a priori concept. Instead, I read the change between draft and policy with an eye for alignment with commenters’ requests (including requests that specific parts of the draft policy do not change.)

My approach of identifying the dimensions of the conflict by comments has benefits and downsides. Compared to other potential measures of success, it is more likely to focus on things that commenters care about. For example, one could measure success by the number of times a comment is mentioned in the agency’s response to comments. However, this may capture strategic responsiveness by agencies choosing to discuss some issues more than others. It also counts explicit rejections toward the measure of responsiveness. One could also measure success by focusing on a-priory potential aspects of the policy. “Lost in the Flood?: Agency Responsiveness to Mass Comment Campaigns in Administrative Rulemaking” (2020) count five factors: (1) the number of regulated entities, (2) number of activities or substances being regulated, (3) the level of pollution standards, (4) the compliance and effective deadlines of the regulation, and (5) the monitoring and reporting requirements. Each takes one value (increasing or decreasing), and each is weighted equally in the analysis. In contrast, starting with comments allows commenters to highlight the issues they care most about.

1.2.1 By organization

Organization-level data sample:

library(tidyverse)
comments_coded %>% 
  dplyr::select(document_id, comment_type, comments, starts_with(c("org_", "coalition")), -org_name_short) %>% 
  group_by(coalition_comment) %>% 
  slice(n = 2) %>%
  kablebox()
document_id comment_type comments org_name org_type org_lead coalition_comment coalition_type coalitions coalition_unopposed coalition_congress coalition_size coalition_position coalition_business coalition_success coalition_leader_success coalition_comments coalition_id
FEMA-2016-0003-0170 org 1 state of alaska gov;state FALSE aasa public 10 FALSE 0 16 1.000000 0 2.0000000 2.0000000 16 1
CFPB-2019-0022-5924 mass 1 NA NA NA aca international private 5 FALSE 0 45 4.311111 36 0.6875000 0.5000000 45 2
WHD-2019-0003-12782 org 1 national association of home builders corp;group FALSE acg private 3 FALSE 1 24 4.000000 0 1.8750000 1.8750000 24 3
NPS-2018-0007-71052 org 1 institute for free speech ngo;advocacy FALSE aclu public 4 FALSE 1 76 1.013158 0 1.9672131 1.9672131 76 4
ICEB-2015-0002-41564 org 1 southeast missouri state university ngo;university FALSE afl-cio public 6 FALSE 0 5 5.000000 2 0.8000000 0.8000000 5 5
FWS-HQ-NWRS-2012-0086-0094 org 1 alaska oil and gas association corp group TRUE alaska oil and gas association private 3 FALSE 0 5 1.800000 3 -0.4000000 -2.0000000 5 7
DEA-2018-0005-1537 org 1 association for accessible medicines corp group FALSE ama public 4 FALSE 0 2 4.000000 1 2.0000000 2.0000000 2 8
BSEE-2018-0002-33478 org 1 la-1 coalition corp group FALSE american petroleum institute private 4 FALSE 0 28 4.500000 22 0.2222222 -2.0000000 28 14
NOAA-NMFS-2013-0101-1828 org 1 forked river tuna club ngo;membership FALSE american sportfishing association private 5 FALSE 0 5 5.000000 1 0.0000000 -2.0000000 1542 17
IRS-2016-0015-0141 mass 1 american federation of government employees gov;federal FALSE americans for tax fairness public 4 FALSE 0 9 4.888889 0 2.0000000 2.0000000 29853 18
WHD-2019-0001-59320 org 1 partnership for medicaid home based care corp;small corp FALSE ancor public 7 FALSE 0 7 5.000000 0 NaN NaN 7 20
CEQ-2019-0003-346818 org 1 cahto tribe gov; tribe FALSE association of american indian affairs private 6 FALSE 0 43 2.790698 0 -1.1428571 -1.1428571 43 23
CFPB-2016-0025-211877 org 1 michigan cbc host committee corp FALSE axcess financial private 6 FALSE 3 109 1.321101 57 -0.6029412 -1.0000000 109 28
PHMSA-2012-0082-0317 org 1 village of elburn gov;local FALSE barrington and illinois trac coalition public 9 FALSE 6 125 4.808000 4 -0.6129032 -0.6129032 125 30
USCBP-2007-0064-0526 org 1 aircraft owners and pilots association ngo;membership;advocacy FALSE bbp public 3 FALSE 2 20 1.550000 7 -1.2000000 -1.2000000 20 32
NOAA-NMFS-2013-0101-1881 org 1 diane marie fishery corp FALSE blue water fishermen’s association private 5 FALSE 0 3 1.666667 2 -2.0000000 -2.0000000 3 34
FWS-HQ-ES-2018-0097-107766 mass 72178 humane society ngo FALSE center for biological diversity public 9 FALSE 0 62 1.193548 0 -1.3255814 -2.0000000 856517 38
NOAA-NMFS-2018-0035-0319 org 1 center for sportfishing policy corp group;coalition TRUE center for sportfishing policy private 5 FALSE 0 6 1.333333 1 -2.0000000 -2.0000000 6 40
TREAS-DO-2007-0015-0032 org 1 center for regulatory effectiveness corp;thinktank FALSE chamber of commerce private 20 FALSE 0 5 1.200000 1 -0.6000000 -0.6000000 5 41
NOAA-NMFS-2008-0096-0019 org 1 united national fishermen’s assoc. corp group;ngo FALSE commercial fishers and processors private 13 FALSE 0 13 2.692308 6 -0.2857143 -0.2857143 13 45
NOAA-NMFS-2018-0035-0329 org 1 blue water fishermen’s association corp group FALSE commercial fishing private 5 FALSE 0 4 3.250000 4 -0.2500000 -0.2500000 4 46
FWS-HQ-ES-2018-0097-57575 org 1 montezuma county gov;local FALSE congressional sportsmen’s foundation private 9 FALSE 0 37 3.864865 2 1.7333333 2.0000000 37 50
TREAS-DO-2007-0015-0112 elected 1 congressman joe pitts congress;joseph r. pitts; mike pence; doug lamborn; joe wilson; phil gingrey; john t. doolittle; wally herger; w. todd akin; marilyn musgrave; bob goodlatte roger wicker; mark souder roscoe bartlett FALSE congressmen public 20 FALSE 1 1 4.000000 0 2.0000000 2.0000000 1 51
CFPB-2019-0006-5717 org 1 coachella valley housing coalition ngo FALSE consumer public 14 FALSE 0 9 1.000000 0 -2.0000000 -2.0000000 9 53
TREAS-DO-2007-0015-0091 org 1 consumer bankers association corp group TRUE consumer bankers association private 20 FALSE 0 2 2.000000 1 2.0000000 2.0000000 2 54
TREAS-DO-2007-0015-0087 org 1 corporate one federal credit union ngo;credit union FALSE credit unions private 20 FALSE 0 2 2.000000 0 -2.0000000 -2.0000000 2 56
FWS-HQ-ES-2018-0007-69520 mass 186 national audubon society ngo;advocacy;ngo;membership FALSE defenders of wildlife public 4 FALSE 0 98 1.122449 0 -1.6938776 -1.6938776 702687 59
USCIS-2010-0017-12455 org 1 department for professional employees, afl-cio ngo;union FALSE dpe afl-cio public 3 FALSE 0 2 1.000000 0 -2.0000000 -2.0000000 2 63
BSEE-2018-0002-46026 mass 14763 defenders of wildlife ngo;advocacy;pressure group FALSE earthjustice public 4 FALSE 3 48 1.041667 0 -1.8947368 -2.0000000 508462 65
USCBP-2007-0064-2108 org 1 bahamas hotel association corp group FALSE ebaa public 3 FALSE 0 5 2.600000 3 -1.2000000 -1.2000000 5 67
CFPB-2019-0006-26784 org 1 the financial clinic corp FALSE empire justice center public 14 FALSE 2 36 1.027778 2 -1.7000000 -2.0000000 36 70
NOAA-NMFS-2008-0096-0085 mass 1 pew ngo;environmental;pressure FALSE environmental community public 13 FALSE 0 20 4.950000 0 -0.9090909 -0.9090909 20 73
FWS-HQ-ES-2018-0097-107766 mass 72178 humane society ngo FALSE environmentalists public 9 FALSE 0 62 1.193548 0 -1.3255814 -1.3255814 856517 75
WHD-2019-0001-0052 org 1 chatmoss country club corp;small corp FALSE epi public 7 FALSE 4 38 1.000000 0 -1.0000000 -1.0000000 38 76
ICEB-2015-0002-40752 org 1 NA ngo;membership;advocacy;pressure group NA fair public 6 FALSE 0 2 1.000000 0 -2.0000000 -2.0000000 2 80
TREAS-DO-2007-0015-0063 mass 1 family research council ngo;faith TRUE family research council public 20 FALSE 0 6 5.000000 0 2.0000000 2.0000000 6 81
FWS-HQ-ES-2018-0097-57575 org 1 montezuma county gov;local FALSE farm bureau private 9 FALSE 0 53 3.943396 17 1.9166667 2.0000000 53 82
FWS-HQ-ES-2018-0097-57575 org 1 montezuma county gov;local FALSE farmers private 9 FALSE 0 53 3.943396 17 1.9166667 1.9166667 53 83
FWS-R9-ES-2008-0093-7786 org 1 minerals management service gov;federal FALSE federal agencies public 4 FALSE 0 2 3.500000 0 -0.5000000 -0.5000000 2 84
NOAA-NMFS-2011-0117-0105 org 1 blue water fishermen’s association corp group FALSE fishing industry private 4 FALSE 0 2 2.000000 2 -2.0000000 -2.0000000 2 86
PHMSA-2012-0082-3151 org 1 george washington university regulatory studies center ngo;university TRUE george washington university regulatory studies center NA 9 FALSE 0 1 3.000000 0 -2.0000000 -2.0000000 1 91
NOAA-NMFS-2008-0096-0031 org 1 north carolina division of marine fisheries gov;state FALSE government public 13 FALSE 0 5 3.800000 0 -1.0000000 -1.0000000 5 92
FWS-HQ-ES-2018-0097-80560 org 1 nez perce tribal executive committee gov;tribe;ej FALSE great lakes indian fish and wildlife commission public 9 FALSE 0 5 1.400000 0 -1.0000000 -1.0000000 5 95
TREAS-DO-2007-0015-0029 org 1 sport view television corporation corp FALSE greyhound racing industry private 20 FALSE 0 7 1.000000 7 -0.8000000 -0.8000000 7 96
NOAA-NMFS-2018-0035-0330 org 1 ocean conservancy ngo;advocacy FALSE gulf restoration network public 5 FALSE 0 18 1.000000 0 -1.9000000 -2.0000000 28488 97
DEA-2018-0005-1133 org 1 vizient corp;membership;health care FALSE hsca public 4 FALSE 0 9 1.666667 2 -1.7777778 -1.7777778 9 98
FWS-HQ-ES-2018-0097-57575 org 1 montezuma county gov;local FALSE hunters private 9 FALSE 0 37 3.864865 2 1.7333333 1.7333333 37 99
FEMA-2016-0003-0200 org 1 kentucky division of emergency management gov;state FALSE iaem-usa public 10 FALSE 0 23 2.000000 0 2.0000000 2.0000000 23 100
CFPB-2019-0006-188547 org 1 credit union national association (cuna) corp group FALSE industry private 14 FALSE 0 5 4.000000 1 2.0000000 2.0000000 5 103
PHMSA-2012-0082-1960 org 1 canadian association of railway suppliers corp group FALSE international public 9 FALSE 0 3 5.000000 1 1.6666667 1.6666667 3 104
OSHA-H005C-2006-0870-1962 org 1 kimberly-clark professional corp;corp FALSE kimberly-clark public 5 FALSE 0 5 3.600000 0 0.2000000 1.0000000 5 107
CEQ-2019-0003-173043 mass 50 liuna NA TRUE liuna private 6 FALSE 0 112 4.526786 1 0.5333333 0.5333333 2884 110
PHMSA-2012-0082-0317 org 1 village of elburn gov;local FALSE local governments public 9 FALSE 6 125 4.808000 4 -0.6129032 -0.6129032 125 111
CFPB-2019-0022-2448 mass 1 comment submitted by barbara welch NA FALSE lshv public 5 FALSE 3 79 1.227848 4 -0.6086957 -0.6086957 79 112
NOAA-NMFS-2008-0096-0074 org 1 environmental defense fund ngo;environmental FALSE marine fish conservation network public 13 FALSE 0 3 5.000000 0 -1.6666667 -1.6666667 3 114
TREAS-DO-2007-0015-0110 org 1 american bankers association corp group FALSE mastercard private 20 FALSE 0 6 1.500000 6 -1.3333333 -1.3333333 6 116
OCC-2020-0026-0256 org 1 innovative lending platform association corp group FALSE mla private 3 FALSE 0 12 3.916667 8 2.0000000 2.0000000 12 122
CFPB-2016-0025-208735 elected 1 senator kimberly lightford illinois senate district 4 FALSE naacp public 6 FALSE 18 70 1.328571 3 0.0317460 2.0000000 70 128
CFPB-2019-0022-9216 org 1 heartland credit union association ngo;credit union FALSE nafcu private 5 FALSE 0 16 3.312500 3 1.3333333 1.3333333 16 129
ICEB-2015-0002-7242 mass 1 NA NA NA nafsa public 6 FALSE 1 108 4.435185 10 1.1607143 1.1607143 108 130
FWS-HQ-ES-2018-0007-50285 org 1 snoqualmie indian tribe gov;tribe;ej FALSE national congress of american indians public 4 FALSE 0 9 1.000000 0 -0.4285714 -0.4285714 9 134
FWS-HQ-ES-2018-0007-51688 org 1 county of siskiyou gov;local FALSE national endangered species act reform coalition private 4 FALSE 0 74 4.148649 36 1.9629630 2.0000000 74 135
FWS-HQ-ES-2018-0006-54226 NA 1 national environmental banking association corp group TRUE national environmental banking association private 5 FALSE 0 1 4.000000 1 NaN NaN 1 136
FWS-HQ-ES-2018-0097-80560 org 1 nez perce tribal executive committee gov;tribe;ej FALSE native americans public 9 FALSE 0 5 1.400000 0 -1.0000000 -1.0000000 5 138
USCIS-2010-0017-12451 org 1 national council of asian pacific americans ngo;coaltion;advocacy FALSE ncapa public 3 FALSE 0 2 4.500000 0 2.0000000 2.0000000 2 139
WHD-2019-0001-59330 org 1 national community pharmacists association corp;group FALSE ncpa private 7 FALSE 0 11 4.000000 0 2.0000000 2.0000000 11 140
FEMA-2016-0003-0253 org 1 louisiana governor’s office of homeland security and emergency preparedness gov;state FALSE ncsl public 10 FALSE 0 2 1.000000 0 2.0000000 2.0000000 2 141
CFPB-2016-0025-211911 org 1 germania credit union ngo;credit union FALSE ncua public 6 FALSE 0 66 2.272727 4 0.0952381 0.0952381 66 142
FEMA-2016-0003-0177 org 1 state of arizona department of emergency and military affairs gov;state FALSE nema public 10 FALSE 0 44 2.886364 2 1.0000000 1.0000000 44 144
OSHA-H005C-2006-0870-1703 elected 1 congressman robert a. brady house-pa FALSE newport news public 5 FALSE 2 30 4.833333 0 1.0357143 1.0357143 30 148
WHD-2019-0001-59231 org 1 national association of convenience stores corp;group FALSE nfib private 7 FALSE 0 1 5.000000 0 NaN NaN 1 149
FEMA-2016-0003-0263 org 1 natural resource defense council/american rivers ngo;advocacy;ej FALSE nrdc public 10 FALSE 0 11 5.000000 3 -2.0000000 -2.0000000 11 155
FEMA-2016-0003-0208 org 1 illinois emergency management agency gov;state FALSE nreca public 10 FALSE 3 36 1.277778 4 2.0000000 2.0000000 36 156
NOAA-NMFS-2011-0117-3359 org 1 shark advocates international and project aware foundation ngo FALSE oceana public 4 FALSE 0 6 5.000000 0 -1.0000000 -2.0000000 12887 157
BSEE-2012-0005-0069 org 1 ipaa corp group FALSE offshore operators committee private 5 FALSE 0 38 1.394737 22 0.8620690 0.0000000 38 158
TREAS-DO-2007-0015-0072 org 1 the depository trust & clearing corporation corp group FALSE other banks private 20 FALSE 0 13 2.384615 9 0.3846154 0.3846154 13 162
CEQ-2019-0003-172067 mass 91604 partnership project NA TRUE partnership project public 6 FALSE 1 135 1.148148 0 -1.7647059 -2.0000000 400081 166
FEMA-2016-0003-0165 org 1 state of iowa gov;state FALSE pew public 10 FALSE 0 223 4.103139 1 -1.8918919 -1.8918919 2982 167
NOAA-NMFS-2013-0101-2237 mass 1 environmental action ngo;pressure group FALSE pew charitable trusts public 5 FALSE 0 132 4.984849 1 -0.5625000 -1.5000000 168292 169
DEA-2018-0005-1555 org 1 pharmaceutical research and manufacturers of america corp group FALSE phrma public 4 FALSE 0 3 3.000000 1 0.6666667 0.6666667 3 170
WHD-2019-0001-59295 org 1 partnership to protect workplace opportunity ngo;advocacy FALSE ppwo public 7 FALSE 1 12 5.000000 0 -2.0000000 -2.0000000 12 171
PHMSA-2012-0082-0327 org 1 dakota gasification company corp FALSE rail industry private 9 FALSE 1 213 1.563380 103 -0.5175439 -0.5175439 213 174
NOAA-NMFS-2008-0096-0018 org 1 recreational fishing alliance ngo;corp group FALSE recreational fishers private 13 FALSE 0 5 3.000000 2 -0.4000000 -0.4000000 5 175
NOAA-NMFS-2018-0035-0319 org 1 center for sportfishing policy corp group;coalition FALSE recreational fishing private 5 FALSE 0 6 1.333333 1 -2.0000000 -2.0000000 6 176
NOAA-NMFS-2008-0096-0012 org 1 south atlantic fishery management council gov;federal;regional FALSE regional fishery management councils public 13 FALSE 1 8 3.000000 0 0.1250000 0.1250000 8 177
BSEE-2013-0011-0007 org 1 shell exploration & production company corp;energy FALSE shell private 4 FALSE 7 90 2.244444 29 -1.2727273 -2.0000000 90 181
WHD-2019-0003-0017 elected 1 congresswoman alma s. adams house-nc FALSE shrm public 3 FALSE 1 25 5.000000 1 0.9600000 0.9600000 25 182
PHMSA-2012-0082-0202 mass 1 sierra club ngo;pressure;membership TRUE sierra club public 9 FALSE 0 47 5.000000 0 -1.2272727 0.0000000 330381 183
NPS-2018-0007-1091 org 1 smithsonian institution ngo;museum FALSE smithsonian public 4 FALSE 0 3 1.333333 0 2.0000000 2.0000000 3 185
OSHA-H005C-2006-0870-1597 org 1 american dental association ngo;professional FALSE southern company public 5 FALSE 0 15 3.000000 0 0.6666667 1.0000000 15 187
BSEE-2012-0005-0027 org 1 state of alaska gov;state TRUE state of alaska public 5 FALSE 0 2 5.000000 0 -1.0000000 -1.0000000 2 190
CFPB-2019-0006-26513 mass 1 california financial service providers (cfsp) NA FALSE thrifty loans private 14 FALSE 0 11 4.363636 7 1.6363636 1.6363636 11 196
PHMSA-2012-0082-1960 org 1 canadian association of railway suppliers corp group FALSE transport canada public 9 FALSE 0 3 5.000000 1 1.6666667 1.6666667 3 197
CFPB-2019-0006-28008 individual 1 results ngo;charity FALSE true public 14 FALSE 0 54 1.648148 2 -1.3400000 -1.3400000 54 198
OFCCP-2014-0004-0071 org 1 international bancshares corporation corp;corp FALSE u.s. chamber of commerce private 4 FALSE 0 13 1.000000 0 1.6923077 2.0000000 13 199
OFCCP-2014-0004-0037 org 1 boston women’s workforce council ngo;advocacy FALSE united brotherhood of carpenters and joiners of america public 4 FALSE 0 7 5.000000 0 -0.5000000 -1.0000000 7 202
MSHA-2011-0001-0140 org 1 appalachian citizens’ law center ngo;legal FALSE united mine workers of america public 3 FALSE 1 4 4.500000 0 0.5000000 -1.0000000 4 203
FEMA-2016-0003-0246 org 1 air worldwide corp FALSE usrc public 10 FALSE 0 11 2.272727 1 1.6363636 1.6363636 11 205
ICEB-2015-0002-41442 org 1 the university of colorado boulder ngo;university FALSE verizon public 6 FALSE 0 3 3.000000 1 0.3333333 1.0000000 3 208
CFPB-2016-0025-211870 org 1 civil justice, inc. ngo;membership;advocacy FALSE wcbc public 6 FALSE 5 148 4.831081 4 0.7627119 0.7627119 148 209
WHD-2019-0001-59303 org 1 world floor covering association corp;group FALSE wfca private 7 FALSE 0 30 4.966667 0 1.5000000 1.5000000 30 211
MSHA-2011-0001-0005 org 1 alamo cement corp;corp FALSE wyoming mining association private 3 FALSE 0 41 2.219512 0 -1.4500000 -2.0000000 41 214
BSEE-2018-0002-19641 org 1 blair field services ltd corp NA NA NA 4 FALSE 0 2 4.000000 2 -2.0000000 -2.0000000 2 NA

Summary counts:

# org comments by type 
comments_coded %>% filter(comment_type == "org") %>%  
  mutate(org_type = str_remove(org_type, ";.*")) %>%
  count(org_type, sort =T) %>% 
  kablebox()
org_type n
ngo 1418
gov 652
corp 452
corp group 438
NA 21
org 8
elected 4
coalition 2
corp groups 2
corp gorup 1
corp grop 1
general assembly-md 1
gov association 1
maryland house of delegates 1
new jersey mayor 1
ngos 1
other 1
senate-md 1 
# org comments by sub-type
comments_coded %>% filter(comment_type == "org") %>%  
  filter(str_detect(org_type, ";")) %>% 
  mutate(org_type_detailed = org_type) %>%
  count(org_type_detailed, sort =T) %>% 
  kablebox()
org_type_detailed n
ngo;advocacy 379
gov;local 205
gov;state 192
corp;group 77
ngo;legal 77
ngo;professional 67
ngo;credit union 61
ngo; advocacy 48
ngo;university 46
gov;tribe 44
gov;tribe;ej 38
corp;corp 37
ngo;coalition 37
ngo;faith 34
gov;federal 32
ngo;union 31
gov;federal;regional 25
ngo;membership 24
ngo;advocacy;membership 19
ngo;thinktank 19
corp group;coalition 17
corp;law firm 17
ngo;membership;advocacy 17
gov;local;coalition 15
corp;bank 14
corp;small corp 14
ngo;legal;advocacy 14
ngo;environmental 13
ngo;pressure group 13
corp;energy 11
corp;legal 11
ngo;philanthropy 9
corp; bank 8
ngo; credit union 8
ngo; environmental advocacy 8
ngo;advocacy;professional 8
ngo;professional;membership 8
ngo;think tank 8
gov; state 7
corp group;energy 6
corp group;professional 6
ngo;advocacy;ngo;membership 6
ngo;advocacy;pressure group 6
ngo;healthcare 6
corp group; trade association 5
corp group;membership 5
corp group;membership;professional 5
gov; county 5
gov; tribe 5
gov;local;county 5
gov;state;coalition 5
ngo;ej 5
ngo;legal;membership 5
ngo;trade association 5
ngo;tribe 5
corp;thinktank 4
gov;local;ej 4
gov;local;ngo 4
gov;local;tribe;ej 4
gov;state agency 4
gov;state;ej 4
ngo; environemental protection 4
ngo;advocacy;coalition 4
ngo;advocacy;legal 4
ngo;membership;pressure 4
ngo;tribe;ej;advocacy 4
corp;busines 3
corp;gov;tribal;ej 3
corp;medical 3
gov; local 3
gov;county 3
gov;local; court 3
ngo; 3
ngo; volunteer 3
ngo;advocacy;ej 3
ngo;advocacy;ngo 3
ngo;corp group 3
ngo;education 3
ngo;federal credit union 3
ngo;financial services 3
ngo;university;legal 3
org;finance 3
coalition;ngo;advocacy 2
corp group; advoacy 2
corp group; bank 2
corp group; chamber of commerce 2
corp group; energy 2
corp group; farmers 2
corp group; natural resoruces; mining association 2
corp group; ranchers 2
corp group;association; livestock 2
corp group;ngo 2
corp group;professional;membership 2
corp;consultants 2
corp;consulting 2
corp;corp;legal 2
corp;finance 2
corp;membership 2
corp;transportation 2
elected;governor 2

1.2.2 By coalition

Coalition-level data sample:

# common names 
coalitions_coded %<>% 
  mutate(coalition = coalition_comment,
         comments = coalition_comments,
         agency = str_remove(docket_id, "-.*"))

coalitions_coded %>% ungroup() %>% group_by(docket_id) %>% 
  slice_max(coalition_comments, n = 2) %>%
  dplyr::select(docket_id, starts_with("coalition")) %>% 
  distinct() %>% 
  kablebox()
docket_id coalition_comment coalition_type coalitions coalition_unopposed coalition_congress coalition_size coalition_position coalition_business coalition_success coalition_leader_success coalition_comments coalition_id Coalition_size Coalition_comments Coalition_Position coalition
BSEE-2012-0005 pew public 5 FALSE 0 4 4.750000 0 1.5000000 1.5000000 1530 197 2-10 More than 100 Supports rule pew
BSEE-2012-0005 offshore operators committee private 5 FALSE 0 32 1.394737 22 0.8620690 0.8620690 38 186 11-100 11-100 Opposes rule offshore operators committee
BSEE-2013-0010 shell private 1 TRUE 0 1 1.000000 0 -2.0000000 -2.0000000 1 213 1 1 Opposes rule shell
BSEE-2013-0011 shell private 4 FALSE 0 88 2.244444 29 -1.2727273 -1.2727273 90 213 11-100 11-100 Opposes rule shell
BSEE-2013-0011 pew public 4 FALSE 0 13 4.692308 0 -0.7500000 -0.7500000 13 197 11-100 11-100 Supports rule pew
BSEE-2017-0008 center for biological diversity public 4 FALSE 0 22 1.041667 0 -2.0000000 -2.0000000 59410 46 11-100 More than 100 Opposes rule center for biological diversity
BSEE-2017-0008 american petroleum institute private 4 FALSE 0 7 3.571429 5 0.2857143 0.2857143 7 17 2-10 2-10 Supports rule american petroleum institute
BSEE-2018-0002 earthjustice public 4 FALSE 0 23 1.074074 0 -1.8947368 -1.8947368 71627 77 11-100 More than 100 Opposes rule earthjustice
BSEE-2018-0002 american petroleum institute private 4 FALSE 0 27 4.500000 22 0.2222222 0.2222222 28 17 11-100 11-100 Supports rule american petroleum institute
CEQ-2019-0003 partnership project public 6 FALSE 1 111 1.172662 0 -1.7647059 -1.7647059 400085 196 More than 100 More than 100 Opposes rule partnership project
CEQ-2019-0003 liuna private 6 FALSE 0 110 4.540323 2 0.3750000 0.3750000 2896 126 More than 100 More than 100 Supports rule liuna
CFPB-2016-0025 wcbc public 6 FALSE 0 141 4.831081 4 0.7627119 0.7627119 148 245 More than 100 More than 100 Supports rule wcbc
CFPB-2016-0025 axcess financial private 6 FALSE 0 106 1.321101 57 -0.6029412 -0.6029412 112 32 More than 100 More than 100 Opposes rule axcess financial
CFPB-2019-0006 true public 14 FALSE 0 52 1.648148 2 -1.3400000 -1.3400000 54 233 11-100 11-100 Opposes rule true
CFPB-2019-0006 true NA 14 FALSE 0 52 1.648148 2 -1.3400000 -1.3400000 54 233 11-100 11-100 Opposes rule true
CFPB-2019-0006 true private 14 FALSE 0 52 1.648148 2 -1.3400000 -1.3400000 54 233 11-100 11-100 Opposes rule true
CFPB-2019-0022 lshv public 5 FALSE 2 70 1.227848 4 -0.6086957 -0.6086957 79 127 11-100 11-100 Opposes rule lshv
CFPB-2019-0022 aca international private 5 FALSE 0 34 4.311111 36 0.6875000 0.6875000 45 2 11-100 11-100 Supports rule aca international
DEA-2018-0005 hsca public 4 FALSE 0 9 1.666667 2 -1.7777778 -1.7777778 9 112 2-10 2-10 Opposes rule hsca
DEA-2018-0005 phrma public 4 FALSE 0 3 3.000000 1 0.6666667 0.6666667 3 200 2-10 2-10 Supports rule phrma
DOI-2015-0005 blanket crew public 6 FALSE 0 14 4.071429 1 2.0000000 2.0000000 14 38 11-100 11-100 Supports rule blanket crew
DOI-2015-0005 support and assist public 6 FALSE 0 11 4.727273 0 2.0000000 2.0000000 11 224 11-100 11-100 Supports rule support and assist
ED-2016-OESE-0032 department of education public 18 FALSE 0 19 3.500000 0 NaN NaN 20 70 11-100 11-100 Supports rule department of education
ED-2016-OESE-0032 public schools public 18 FALSE 0 14 5.000000 0 NaN NaN 15 203 11-100 11-100 Supports rule public schools
ED-2016-OESE-0032 public schools NA 18 FALSE 0 14 5.000000 0 NaN NaN 15 203 11-100 11-100 Supports rule public schools
FEMA-2016-0003 pew public 10 FALSE 0 217 4.103139 1 -1.8918919 -1.8918919 2982 197 More than 100 More than 100 Supports rule pew
FEMA-2016-0003 nema public 10 FALSE 0 38 2.886364 2 1.0000000 1.0000000 44 169 11-100 11-100 Opposes rule nema
FWS-HQ-ES-2018-0006 defenders of wildlife public 5 FALSE 0 88 1.138889 0 -1.6666667 -1.6666667 440844 69 11-100 More than 100 Opposes rule defenders of wildlife
FWS-HQ-ES-2018-0006 national endangered species act reform coalition private 5 FALSE 0 76 4.197368 36 0.2222222 0.2222222 76 157 11-100 11-100 Supports rule national endangered species act reform coalition
FWS-HQ-ES-2018-0007 defenders of wildlife public 4 FALSE 0 84 1.122449 0 -1.6938776 -1.6938776 702687 69 11-100 More than 100 Opposes rule defenders of wildlife
FWS-HQ-ES-2018-0007 national endangered species act reform coalition private 4 FALSE 0 73 4.148649 36 1.9629630 1.9629630 74 157 11-100 11-100 Supports rule national endangered species act reform coalition
FWS-HQ-ES-2018-0097 center for biological diversity public 7 FALSE 0 39 1.193548 0 -1.3255814 -1.3255814 856518 46 11-100 More than 100 Opposes rule center for biological diversity
FWS-HQ-ES-2018-0097 farm bureau private 7 FALSE 0 45 3.943396 17 1.9166667 1.9166667 53 94 11-100 11-100 Supports rule farm bureau
FWS-HQ-NWRS-2012-0086 defenders of wildlife public 3 FALSE 0 13 4.692308 0 0.6363636 0.6363636 53271 69 11-100 More than 100 Supports rule defenders of wildlife
FWS-HQ-NWRS-2012-0086 alaska oil and gas association private 3 FALSE 0 5 1.800000 3 -0.4000000 -0.4000000 5 9 2-10 2-10 Opposes rule alaska oil and gas association
FWS-R9-ES-2008-0093 earthjustice public 4 FALSE 0 47 1.051724 0 -1.7826087 -1.7826087 58 77 11-100 11-100 Opposes rule earthjustice
FWS-R9-ES-2008-0093 national endangered species act reform coalition private 4 FALSE 0 21 4.333333 8 0.2857143 0.2857143 21 157 11-100 11-100 Supports rule national endangered species act reform coalition
ICEB-2015-0002 nafsa public 6 FALSE 0 56 4.435185 10 1.1607143 1.1607143 109 148 11-100 More than 100 Supports rule nafsa
ICEB-2015-0002 afl-cio public 6 FALSE 0 5 5.000000 2 0.8000000 0.8000000 5 6 2-10 2-10 Supports rule afl-cio
IRS-2016-0015 americans for tax fairness public 4 FALSE 0 9 4.888889 0 2.0000000 2.0000000 29853 21 2-10 More than 100 Supports rule americans for tax fairness
IRS-2016-0015 congressmen public 4 FALSE 0 1 4.000000 0 -2.0000000 -2.0000000 1 61 1 1 Supports rule congressmen
IRS-2016-0015 small businesses public 4 FALSE 0 1 4.000000 0 2.0000000 2.0000000 1 216 1 1 Supports rule small businesses
IRS-2016-0015 swiss re private 4 FALSE 0 1 3.000000 0 -1.0000000 -1.0000000 1 226 1 1 Supports rule swiss re
MSHA-2011-0001 wyoming mining association private 3 FALSE 0 38 2.219512 0 -1.4500000 -1.4500000 41 251 11-100 11-100 Opposes rule wyoming mining association
MSHA-2011-0001 united mine workers of america public 3 FALSE 0 4 4.500000 0 0.5000000 0.5000000 4 239 2-10 2-10 Supports rule united mine workers of america
NOAA-NMFS-2008-0096 environmental community public 13 FALSE 0 15 4.950000 0 -0.9090909 -0.9090909 20 86 11-100 11-100 Supports rule environmental community
NOAA-NMFS-2008-0096 commercial fishers and processors private 13 FALSE 0 13 2.692308 6 -0.2857143 -0.2857143 13 55 11-100 11-100 Opposes rule commercial fishers and processors
NOAA-NMFS-2011-0117 oceana public 4 FALSE 0 5 5.000000 0 -1.0000000 -1.0000000 12887 185 2-10 More than 100 Supports rule oceana
NOAA-NMFS-2011-0117 fishing industry private 4 FALSE 0 2 2.000000 2 -2.0000000 -2.0000000 2 98 2-10 2-10 Opposes rule fishing industry
NOAA-NMFS-2011-0117 blue water fishermen’s association private 4 FALSE 0 2 2.000000 2 -2.0000000 -2.0000000 2 40 2-10 2-10 Opposes rule blue water fishermen’s association
NOAA-NMFS-2012-0059 pew public 6 FALSE 0 31 1.000000 1 -1.6875000 -1.6875000 102875 197 11-100 More than 100 Opposes rule pew
NOAA-NMFS-2012-0059 fishing industry private 6 FALSE 0 12 4.833333 7 -1.1666667 -1.1666667 12 98 11-100 11-100 Supports rule fishing industry
NOAA-NMFS-2012-0059 west coast seafood processors association private 6 FALSE 0 12 4.833333 7 -1.1666667 -1.1666667 12 246 11-100 11-100 Supports rule west coast seafood processors association
NOAA-NMFS-2013-0050 pew public 8 FALSE 0 8 1.000000 1 -1.3333333 -1.3333333 74818 197 2-10 More than 100 Opposes rule pew
NOAA-NMFS-2013-0050 commercial fishing private 8 FALSE 0 6 3.571429 5 0.8571429 0.8571429 7 56 2-10 2-10 Supports rule commercial fishing
NOAA-NMFS-2013-0050 northeast seafood coalition private 8 FALSE 0 6 3.571429 5 0.8571429 0.8571429 7 180 2-10 2-10 Supports rule northeast seafood coalition
NOAA-NMFS-2013-0101 pew charitable trusts public 5 FALSE 0 60 4.984849 1 -0.5625000 -0.5625000 168293 199 11-100 More than 100 Supports rule pew charitable trusts
NOAA-NMFS-2013-0101 american sportfishing association private 5 FALSE 0 5 5.000000 1 0.0000000 0.0000000 1542 20 2-10 More than 100 Supports rule american sportfishing association
NOAA-NMFS-2018-0035 gulf restoration network public 5 FALSE 0 11 1.000000 0 -1.9000000 -1.9000000 28488 110 11-100 More than 100 Opposes rule gulf restoration network
NOAA-NMFS-2018-0035 recreational fishing private 5 FALSE 0 6 1.333333 1 -2.0000000 -2.0000000 6 206 2-10 2-10 Opposes rule recreational fishing
NOAA-NMFS-2018-0035 center for sportfishing policy private 5 FALSE 0 6 1.333333 1 -2.0000000 -2.0000000 6 48 2-10 2-10 Opposes rule center for sportfishing policy
NOAA-NMFS-2018-0035-0326 gulf restoration network public 1 TRUE 0 1 1.000000 0 NaN NaN 652 110 1 More than 100 Opposes rule gulf restoration network
NOAA-NOS-2013-0091 national marine sanctuary foundation public 4 FALSE 0 16 4.263158 1 2.0000000 2.0000000 19 159 11-100 11-100 Supports rule national marine sanctuary foundation
NOAA-NOS-2013-0091 ocean industries private 4 FALSE 0 2 2.000000 2 -2.0000000 -2.0000000 2 184 2-10 2-10 Opposes rule ocean industries
NOAA-NOS-2013-0091 american petroleum institute private 4 FALSE 0 2 2.000000 2 -2.0000000 -2.0000000 2 17 2-10 2-10 Opposes rule american petroleum institute
NPS-2018-0007 aclu public 4 FALSE 1 57 1.013158 0 1.9672131 1.9672131 76 4 11-100 11-100 Opposes rule aclu
NPS-2018-0007 smithsonian public 4 FALSE 0 3 1.333333 0 2.0000000 2.0000000 3 217 2-10 2-10 Opposes rule smithsonian
OCC-2020-0026 aclu public 3 FALSE 0 74 1.000000 0 -1.9523810 -1.9523810 222 4 11-100 More than 100 Opposes rule aclu
OCC-2020-0026 mla private 3 FALSE 0 12 3.916667 8 2.0000000 2.0000000 12 139 11-100 11-100 Supports rule mla
OFCCP-2014-0004 aclu public 4 FALSE 0 11 4.000000 0 -2.0000000 -2.0000000 7152 4 11-100 More than 100 Supports rule aclu
OFCCP-2014-0004 u.s. chamber of commerce private 4 FALSE 0 13 1.000000 0 1.6923077 1.6923077 13 234 11-100 11-100 Opposes rule u.s. chamber of commerce
OSHA-H005C-2006-0870 newport news public 5 FALSE 0 22 4.833333 0 1.0357143 1.0357143 30 173 11-100 11-100 Supports rule newport news
OSHA-H005C-2006-0870 southern company public 5 FALSE 0 15 3.000000 0 0.6666667 0.6666667 15 219 11-100 11-100 Supports rule southern company
PHMSA-2012-0082 sierra club public 7 FALSE 0 24 5.000000 0 -1.2272727 -1.2272727 330381 215 11-100 More than 100 Supports rule sierra club
PHMSA-2012-0082 american petroleum institute private 7 FALSE 0 209 1.563380 104 -0.4956522 -0.4956522 214 17 More than 100 More than 100 Opposes rule american petroleum institute
TREAS-DO-2007-0015 other banks private 20 FALSE 0 13 2.384615 9 0.3846154 0.3846154 13 191 11-100 11-100 Opposes rule other banks
TREAS-DO-2007-0015 greyhound racing industry private 20 FALSE 0 7 1.000000 7 -0.8000000 -0.8000000 7 109 2-10 2-10 Opposes rule greyhound racing industry
USCBP-2007-0064 bbp public 3 FALSE 0 20 1.550000 7 -1.2000000 -1.2000000 23 36 11-100 11-100 Opposes rule bbp
USCBP-2007-0064 ebaa public 3 FALSE 0 5 2.600000 3 -1.2000000 -1.2000000 5 79 2-10 2-10 Opposes rule ebaa
USCG-2010-0990 boatus public 3 FALSE 0 5 2.333333 2 -1.0000000 -1.0000000 6 41 2-10 2-10 Opposes rule boatus
USCG-2010-0990 nmma public 3 FALSE 0 3 4.000000 1 2.0000000 2.0000000 4 177 2-10 2-10 Supports rule nmma
USCIS-2010-0017 ncapa public 3 FALSE 0 2 4.500000 0 2.0000000 2.0000000 2 164 2-10 2-10 Supports rule ncapa
USCIS-2010-0017 dpe afl-cio public 3 FALSE 0 2 1.000000 0 -2.0000000 -2.0000000 2 75 2-10 2-10 Opposes rule dpe afl-cio
WHD-2011-0001 congresswoman lucille roybal-allard public 3 FALSE 0 1 4.000000 0 NaN NaN 1 62 1 1 Supports rule congresswoman lucille roybal-allard
WHD-2011-0001 governor terry e. branstad public 3 FALSE 0 1 1.000000 0 NaN NaN 1 106 1 1 Opposes rule governor terry e. branstad
WHD-2019-0001 epi public 7 FALSE 0 38 1.000000 0 -1.0000000 -1.0000000 38 88 11-100 11-100 Opposes rule epi
WHD-2019-0001 wfca private 7 FALSE 0 30 4.966667 0 1.5000000 1.5000000 30 248 11-100 11-100 Supports rule wfca
WHD-2019-0003 epi public 3 FALSE 0 40 1.000000 0 -1.9756098 -1.9756098 44391 88 11-100 More than 100 Opposes rule epi
WHD-2019-0003 shrm public 3 FALSE 0 23 5.000000 1 0.9600000 0.9600000 25 214 11-100 11-100 Supports rule shrm

Histograms of coalition variables

d <- coalitions_coded 

ggplot(d, aes(x = coalition_success)) + 
  geom_histogram() + 
  labs(x = "Coalition Success")

ggplot(d, aes(x = coalition_size)) + 
  geom_histogram() + 
  labs(x = "Coalition size")
Hand-coded Data by CoalitionHand-coded Data by Coalition

Figure 1.2: Hand-coded Data by Coalition

Number of comments

#TODO
#ggplot(d, aes( x= comment_length)) + geom_histogram()+ labs(x = "% (Comment length/proposed rule length)*100")
ggplot(d, aes( x= log(comments))) + 
  geom_histogram() + 
  labs(x = "Log(comments)")
Number of Comments Linked to Hand-Coded Coalitions

Figure 1.3: Number of Comments Linked to Hand-Coded Coalitions

Coalitions by type (public interest vs. private interest)

ggplot(d %>% drop_na(coalition_type)) + 

    aes(x = as.numeric(coalition_business)) + 
  geom_histogram(stat = "count") + 
  labs(x = "Businesses per coalition",
       title = "Number of Businesses by Coalition Type") + 
  facet_wrap("coalition_type")#, scales = "free_x")

ggplot(coalitions_coded %>% filter(!is.na(coalition_type)), aes(x = Coalition_size)) + 
  geom_histogram(stat = "count")+ 
  labs(x = "Coalition Size",
       title = "Coalition Size by Coalition Type") +
  facet_wrap("coalition_type", scales = "free_x")

ggplot(coalitions_coded %>% filter(!is.na(coalition_type)), aes(x = Comments)) + 
  geom_histogram(stat = "count")+ 
  labs(x = "Total Number of Comments",
       title = "Number of Comments by Coalition Type") +
  facet_wrap("coalition_type", scales = "free_x")

1.2.2.1 Number of supportive comments

coalitions_coded %>% 
  drop_na(coalition_type, Coalition_Position) %>% 
  distinct(coalition_comment, coalition_type, coalition_size, coalition_success, comments, agency, Coalition_Position) %>% 
  ungroup() %>%
  ggplot() +
  aes(y = coalition_success, x = log(comments), color = coalition_type) +
  geom_jitter(aes(size = coalition_size), alpha = .5) +
  geom_smooth(se = FALSE, method = "lm")+ 
  facet_grid(Coalition_Position ~ .)

coalitions_coded %>% 
  drop_na(coalition_type, Coalition_Position) %>% 
  distinct(coalition_comment, coalition_type, coalition_size, coalition_success, comments, agency, president, Coalition_Position) %>% 
  mutate(comments = comments) %>% 
  ungroup() %>%
  ggplot() +
  aes(y = coalition_success, x = log(comments), color = coalition_type) +
  geom_jitter(aes(size = coalition_size), alpha = .5) +
  geom_smooth(se = FALSE, method = "lm") + 
  #facet_wrap("president")  + 
  facet_grid(Coalition_Position ~ president)
Lobbying Success by Number of Supportive CommentsLobbying Success by Number of Supportive Comments

Figure 1.4: Lobbying Success by Number of Supportive Comments

1.2.2.2 Coalition Size

(number of supportive organizations)

coalitions_coded %>% 
  drop_na(coalition_type, Coalition_Position) %>% 
  distinct(coalition_comment, coalition_type, coalition_size, coalition_success, comments, agency, president, Coalition_Position) %>% 
  ungroup() %>%
  ggplot() +
  aes(y = coalition_success, x = coalition_size, color = coalition_type) +
  geom_jitter(aes(size = comments ), alpha = .5) +
  geom_smooth(se = FALSE, method = "lm") + 
  facet_grid(Coalition_Position ~ .)+ 
  scale_size_continuous(labels = comma)

coalitions_coded %>% 
  drop_na(coalition_type, Coalition_Position) %>% 
  distinct(coalition_comment, coalition_type, coalition_size, coalition_success, comments, agency, president, Coalition_Position) %>% 
  ungroup() %>%
  ggplot() +
  aes(y = coalition_success, x = coalition_size, color = coalition_type) +
  geom_jitter(aes(size = comments ), alpha = .5) +
  geom_smooth(se = FALSE, method = "lm") + 
  #facet_wrap("president") + 
  facet_grid(Coalition_Position ~ president)+ 
  scale_size_continuous(labels = comma)
Lobbying Success by Number of Supportive CommentsLobbying Success by Number of Supportive Comments

Figure 1.5: Lobbying Success by Number of Supportive Comments

1.3 Machine-coded Data

IN PROGRESS

Dependent variable: The percent change in policy text…

Explanatory variables: The total number of comments…

1.4 Comments from legislators

One mechanism by which campaigns may influence policy is by mobilizing members of Congress. Thus, I identify comments submitted by members of Congress and count the number of legislators in each lobbying coalition.

load(here::here("data", "comments_congress.Rdata"))

comments_congress$Year %<>% as.numeric()

breaks <- seq(2000, 2020,by = 2)

comments_congress %>% 
  as_tibble() %>%
  filter(Year %>% as.numeric() > 2000,
         Year %>% as.numeric() < 2021) %>% 
  add_count(agency, name = "agency_n") %>%
  filter(agency_n > 88) %>% 
  count(Year, Chamber, agency, sort = TRUE) %>%
  ggplot() +
  aes(x = Year, y = n, fill = Chamber) + 
  geom_col(position = "stack") + 
  facet_wrap("agency", scales = "free") + 
  labs(x = "" ,
       y = "Number of Rulemaking Comments from Members of Congress") + 
  scale_x_continuous(breaks = breaks) + 
  theme(axis.text.x = element_text(angle = 90),
        axis.ticks.x = element_blank(),
        panel.grid.major.x = element_blank())

# table 
# elected comments by type
comments_coded %>% 
  filter(comment_type == "elected") %>%  
  mutate(org_type = str_remove(org_type, "-.*|;.*| .*")) %>%
  count(org_type,  sort =T) %>% 
  rename(elected_type = org_type) %>%
  kablebox()
elected_type n
house 26
senate 16
congress 5
gov 5
florida 4
governor 4
maryland 4
mayor 4
mississippi 4
representative 4
alaska 3
illinois 3
senator 3
senators 3
texas 3
NA 3
california 2
city 2
oklahoma 2
state 2
attorney 1
baltimore 1
berkeley 1
carbondale 1
carver 1
elected 1
georgia 1
iowa 1
jersey 1
kentucky 1
linda 1
ma 1
member 1
missouri 1
mt 1
nebraska 1
new 1
north 1
ohio 1
pennsylvania 1
rosemead 1
santa 1
south 1
tennessee 1
Number of Letters from Members of Congress Received During Rulemaking per Year

Figure 1.6: Number of Letters from Members of Congress Received During Rulemaking per Year

2 Descriptives

Hypothesis 2.1 Most people engage in national policy processes as a result of organized public pressure campaigns.

Hand-coded sample [in progress]

comments_coded %>% 
  # split out mass comments submitted with org comments
  mutate(comment_type = ifelse(comment_type == "org" & comments > 99,
         "org;mass", 
         comment_type) %>% 
  str_split(";")) %>% 
  unnest(comment_type) %>% 
  mutate(comments = ifelse(comment_type == "org",
                           1, 
                           comments)) %>% 
  group_by(comment_type) %>% 
  tally(comments) %>% 
  kablebox()
comment_type n
coalition 1
elected 126
individual 531
mass 3395900
org 3006
NA 0

Full data [in progress]

# massive undercount of mass
#TODO include new, further collapsed data
comments_min %>% 
  mutate(mass = number_of_comments_received > 99) %>% 
  group_by(mass) %>% 
  tally(number_of_comments_received)%>% 
  kablebox()
mass n
FALSE 13639400
TRUE 69515059
Hypothesis 2.2 Public pressure campaigns are organized by coalitions that include groups that engage in sophisticated technical lobbying.

Nearly all mass comments in the hand-coded rules were mobilized by a group that also engaged in sophisticated lobbying.

Organizations (if any) affilitated with different types of comments in the hand-coded data:

comments_coded %>% 
  group_by(docket_id, coalition, coalition_comments, coalition_type) %>% 
  mutate(comment_type = comment_type %>% as.factor()) %>%
  count(comment_type) %>%
  ungroup() %>% 
  drop_na(comment_type) %>% 
  filter(!comment_type %in% c("NA", "coalition", "mass")) %>% 
  pivot_wider(names_from =  comment_type, values_from = n) %>%
  arrange(-coalition_comments) %>% 
  kablebox()
docket_id coalition coalition_comments coalition_type org elected individual
FWS-HQ-ES-2018-0097 center for biological diversity 856518 public 44 NA 8
FWS-HQ-ES-2018-0007 defenders of wildlife 702687 public 73 NA 1
FWS-HQ-ES-2018-0006 defenders of wildlife 440844 public 78 NA 1
CEQ-2019-0003 partnership project 400085 public 96 2 8
PHMSA-2012-0082 sierra club 330381 public 22 NA NA
NOAA-NMFS-2013-0101 pew charitable trusts 168293 public 18 NA NA
NOAA-NMFS-2012-0059 pew 102875 public 16 NA NA
NOAA-NMFS-2013-0050 pew 74818 public 9 NA NA
BSEE-2018-0002 earthjustice 71627 public 16 3 1
BSEE-2017-0008 center for biological diversity 59410 public 17 2 2
FWS-HQ-NWRS-2012-0086 defenders of wildlife 53271 public 11 NA NA
WHD-2019-0003 epi 44391 public 36 5 NA
IRS-2016-0015 americans for tax fairness 29853 public 1 NA NA
NOAA-NMFS-2018-0035 gulf restoration network 28488 public 10 NA NA
NOAA-NMFS-2011-0117 oceana 12887 public 5 NA NA
OFCCP-2014-0004 aclu 7152 public 8 NA NA
PHMSA-2012-0082 NA 3084 NA 3 NA NA
FEMA-2016-0003 pew 2982 public 37 NA 185
CEQ-2019-0003 liuna 2896 private 120 1 NA
NOAA-NMFS-2013-0101 american sportfishing association 1542 private 4 NA NA
BSEE-2012-0005 pew 1530 public 2 1 NA
OCC-2020-0026 aclu 222 public 70 7 NA
PHMSA-2012-0082 american petroleum institute 214 private 112 5 NA
ICEB-2015-0002 NA 183 NA NA NA 182
CFPB-2016-0025 wcbc 148 public 118 7 7
CFPB-2019-0006 NA 134 NA 52 9 1
PHMSA-2012-0082 barrington and illinois trac coalition 125 public 115 9 1
CFPB-2016-0025 axcess financial 112 private 68 4 11
ICEB-2015-0002 nafsa 109 public 54 1 2
CFPB-2019-0006 true 108 private 3 NA NA
CFPB-2019-0006 true 108 public 46 NA 1
CFPB-2019-0006 true 108 NA 1 NA NA
BSEE-2013-0011 shell 90 private 28 4 NA
CFPB-2019-0022 lshv 79 public 57 2 2
FWS-HQ-ES-2018-0006 national endangered species act reform coalition 76 private 75 1 NA
NPS-2018-0007 aclu 76 public 52 1 7
FWS-HQ-ES-2018-0007 national endangered species act reform coalition 74 private 73 NA NA
CFPB-2019-0006 empire justice center 72 public 25 4 1
CFPB-2016-0025 naacp 70 public 31 22 2
CFPB-2016-0025 ncua 66 public 65 NA NA
WHD-2019-0001 NA 66 NA 62 3 NA
FWS-R9-ES-2008-0093 earthjustice 58 public 45 NA 13
FWS-HQ-ES-2018-0097 farm bureau 53 private 47 1 2
CFPB-2019-0022 aca international 45 private 42 NA NA
FEMA-2016-0003 nema 44 public 44 NA NA
CEQ-2019-0003 association of american indian affairs 43 private 43 NA NA
MSHA-2011-0001 wyoming mining association 41 private 41 NA NA
BSEE-2012-0005 offshore operators committee 38 private 23 NA NA
WHD-2019-0001 epi 38 public 33 5 NA
FWS-HQ-ES-2018-0097 congressional sportsmen’s foundation 37 private 31 1 2
FEMA-2016-0003 nreca 36 public 33 3 NA
OSHA-H005C-2006-0870 newport news 30 public 26 2 NA
WHD-2019-0001 wfca 30 private 30 NA NA
BSEE-2018-0002 american petroleum institute 28 private 17 NA NA
FWS-HQ-ES-2018-0097 hunters 25 private 20 1 1
WHD-2019-0003 shrm 25 public 24 1 NA
WHD-2019-0003 acg 24 private 23 1 NA
FEMA-2016-0003 iaem-usa 23 public 23 NA NA
USCBP-2007-0064 bbp 23 public 18 2 3
CFPB-2019-0006 thrifty loans 22 private 10 NA NA
CFPB-2016-0025 NA 21 NA 14 NA 5
FWS-R9-ES-2008-0093 national endangered species act reform coalition 21 private 21 NA NA
ED-2016-OESE-0032 department of education 20 public 2 NA NA
NOAA-NMFS-2008-0096 environmental community 20 public 11 NA NA
NOAA-NOS-2013-0091 national marine sanctuary foundation 19 public 15 1 3
CFPB-2019-0006 consumer 18 public 8 NA NA
CFPB-2019-0022 nafcu 16 private 16 NA NA
FEMA-2016-0003 aasa 16 public 16 NA NA
ED-2016-OESE-0032 public schools 15 public 2 NA NA
ED-2016-OESE-0032 NA 15 public 1 NA NA
ED-2016-OESE-0032 NA 15 NA 13 1 NA
OSHA-H005C-2006-0870 southern company 15 public 15 NA NA
DOI-2015-0005 blanket crew 14 public 12 1 1
USCIS-2010-0017 NA 14 NA 1 NA 13
BSEE-2013-0011 pew 13 public 12 NA 1
FWS-HQ-ES-2018-0006 national congress of american indians 13 public 13 NA NA
NOAA-NMFS-2008-0096 commercial fishers and processors 13 private 9 NA NA
NPS-2018-0007 NA 13 NA NA NA 13
OFCCP-2014-0004 u.s. chamber of commerce 13 private 13 NA NA
TREAS-DO-2007-0015 other banks 13 private 13 NA NA
FEMA-2016-0003 NA 12 NA NA NA 12
NOAA-NMFS-2012-0059 fishing industry 12 private 12 NA NA
NOAA-NMFS-2012-0059 west coast seafood processors association 12 private 12 NA NA
OCC-2020-0026 mla 12 private 12 NA NA
WHD-2019-0001 ppwo 12 public 11 1 NA
DOI-2015-0005 support and assist 11 public 10 NA NA
FEMA-2016-0003 nrdc 11 public 11 NA NA
FEMA-2016-0003 usrc 11 public 11 NA NA
WHD-2019-0001 ncpa 11 private 11 NA NA
CFPB-2019-0006 industry 10 private 4 NA NA
ED-2016-OESE-0032 national education association 10 public 2 NA NA
DEA-2018-0005 hsca 9 public 9 NA NA
FWS-HQ-ES-2018-0007 national congress of american indians 9 public 9 NA NA
NOAA-NMFS-2012-0059 new england fishery management council 9 public 9 NA NA
NOAA-NMFS-2012-0059 regional councils 9 public 9 NA NA
USCBP-2007-0064 NA 9 NA 9 NA NA
DOI-2015-0005 big autonomy 8 public 7 NA 1
NOAA-NMFS-2008-0096 regional fishery management councils 8 public 7 1 NA
WHD-2019-0001 ancor 8 private 8 NA NA
BSEE-2017-0008 american petroleum institute 7 private 7 NA NA
Hypothesis 2.3 Public interest group coalitions mobilize more often than private interest group (e.g., business-led) coalitions.

Yes, but not as much as I expected.

Hypothesis 2.4 Public interest group coalitions mobilize more successfully than private interest group (e.g., business-led) coalitions.

Yes, by far.

Hypothesis 2.5 Public pressure campaigns targeting national policy are most often run by large national policy advocacy organizations.

Yes.

Hypothesis 2.6 If narrow private interest groups (e.g., businesses) launch public pressure campaigns, it is a response to an opposing campaign.

Yes.

3 Models of influence/success

3.1 DV = Comments from members of Congress

Hypothesis 3.1 The scale of public engagement moderates elected officials’ engagement in agency rulemaking engagement.

The simplest model of the relationship between congressional attention and public attention would be to model the count of legislator letters as a function of features of the rulemaking, including the total number of public comments. The number of letters from members of congress would be a count process; this would be a Poisson or negative binomial regression.

In equation (3.1), \(y_{j}\) is a count of the number of legislator comments on a proposed rule \(j\), \(\beta_1\) is the effect of a one-unit increase in the logged number of public comments on proposed rule \(j\), and \(\eta\) is a vector of coefficients on other factors (\(X_{j}\)) that may lead legislators to comment.

\[ y_{j}= \beta_0 + \beta_1 \text{log(Public comments)}_{j} + \eta X_{j} + \epsilon_{j} \tag{3.1} \]

Alternatively, if we want to control for legislator characteristics that may make them more or less likely to comment on a rule, we can make members of Congress the unit of analysis. The dependent variable is now whether or not a given legislator \(i\) commented on proposed rule \(j\). The relationship between public engagement and legislator engagement can be modeled by Equation (3.2), where \(Pr(Comment_{ij})\) is the probability that legislator \(i\) comments on a proposed rule \(j\), \(\beta_1\) is the effect of a one-unit increase in the logged number of public comments on proposed rule \(j\), and \(\eta\) is a vector of coefficients on other factors (\(X_{ij}\)) that may affect whether a legislator engages.

\[logit(Pr(\text{Legislator comment}_{ij})) = \beta_0 + \beta_1 \text{log(Public comments)}_{ij} + \eta X_{ij} + \epsilon_{ij} \tag{3.2}\]

Hypothesis 3.2 Public pressure campaigns attract oversight from allies. The more comments supporting a position, the more likely principals holding that position are to engage.
Hypothesis 3.3 Public pressure campaigns reduce oversight from opponents. The more comments opposing a position, the less likely principals holding that position are to engage.

The simplest model of the relationship between congressional attention and public support or opposition to a proposed rule would be to model the net count of legislator letters supporting and opposing the proposed as a function of features of the rulemaking, including the net number of public comments supporting and opposing. As the number of letters from members of congress would be a count process, this would be Poisson or negative binomial regression.

The model is the same as equation (3.1) except that \(y_{j}\) is now the net number of legislator comments supporting a proposed rule \(j\), and \(\beta_1\) is now the effect of a one-unit increase in the logged net number of public comments supporting proposed rule \(j\).

With a measure of the likely position on each rule (for example, if promulgated by a co-partisan administration), the individual legislator can be the unit of analysis. The probability that legislator \(i\) will comment on rule \(j\), given their position \(p_{ij}\) on a proposed rule \(j\) (\(Pr(Comment_{ij}i|p_{ij})\)), is modeled in equation (3.3). Hypothesis 3.2 implies that \(\beta_1\) is positive and Hypothesis 3.3 implies that \(\beta_2\) is negative.

\[logit(Pr(\text{Legislator Comment}_{ij}|p_{ij})) = \beta_0 + \beta_1 \text{Comments supporting } p_{ij} + \beta_2 \text{Comments opposing } p_{ij} + \eta X_{ij} + \epsilon_{ij} \tag{3.3}\]

3.2 DV = Lobbying success

For all three measures of lobbying success, I assess the relationship between lobbying success and mass comments by modeling coalition \(i\)’s lobbying success in a rulemaking \(j\), \(y_{ij}\) as a combination of whether the coalition is unopposed, the coalition’s size, whether it is a business coalition, and the logged number of mass comments. I estimate these relationships using OLS regression.

\(Y_{ij} = \boldsymbol{\beta}_1 \textbf{log(Comments)}_{ij} + {\beta}_2\text{Size}_{ij} + {\beta}_3\text{Unopposed}_{ij} + {\beta}_3\text{Coalition Type}_{ij} + \epsilon_{ij}\)

I use two related measures of coalition type. Models 1 and 3 use my classification of coalitions as primarily public or private interests. Models 2 and 4 below use a related measure: the share of coalition members that are businesses or trade associations. Models 3 and 4 include interacting each measure of the coalition’s type with a dummy for president Trump rather than President Obama’s administration. Bush-era rules are dropped from these models for simplicity.

Challenges for inference

Non-independence

Organzations lobbying in coalitions

The hand-coded sample includes 4,577 hand-coded documents representing 3,399,682 comments. However, many of these comments belong coalitions and are thus not independent. When Friends of Earth and the Sierra Club lobbying together on a rule, the success of each depends on the other. Thus, I group comments into coalitions. The hand-coded sample includes 291 “coalitions,” 141 of which are single organizations (not really coalitions), leaving 150 true coalitions of groups lobbing together.

3.2.0.1 Coalitions lobbying on rules

The fact that several coalitions may lobby on the same rule creates a lesser form of dependence among observations. One coalition’s lobbying success is correlated with another coalition’s lobbying success to the extent that they are asking for the same or contradicting things. Because we have grouped organizations into coalitions, the causally-related asks (those organizations lobbying because another organization is) are largely accounted for.

4 Results

4.0.1 DV = Coalition success

Note: these models include coalitions of 1 (organizations lobbying alone), but results are similar if I exclude them, except that coalition size has a much weaker correlation with success.

NOTE: At this time, the sample mostly rules that received an unusual number of comments, so these results are based on variation with high-salience rulemakings.

m_business <- lm(coalition_success ~ 
          log(comments) + 
          #comment_length + 
          coalition_business +  
          log(coalition_size) + 
          coalition_unopposed, 
        data = coalitions_coded) 

m <- lm(coalition_success ~ 
          log(comments) + 
          #comment_length + 
          coalition_type +  
          log(coalition_size) + 
          coalition_unopposed, 
        data = coalitions_coded) 

m_business_president <- lm(coalition_success ~ 
          log(comments) + 
          #comment_length + 
          coalition_business*president +  
          log(coalition_size) + 
          coalition_unopposed, 
        data = coalitions_coded %>% filter(president != "Bush")) 

m_president <- lm(coalition_success ~ 
          log(comments) + 
          #comment_length + 
          coalition_type*president +  
          log(coalition_size) + 
          coalition_unopposed, 
        data = coalitions_coded %>% filter(president != "Bush")) 

models <- list(m, 
               m_business, 
               m_president,
               m_business_president)

modelsummary(models, stars = TRUE)
Model 1 Model 2 Model 3 Model 4
(Intercept) -0.377* -0.288** -0.645** 0.050
(0.194) (0.132) (0.318) (0.194)
log(comments) -0.152*** -0.138*** -0.147*** -0.141***
(0.052) (0.052) (0.051) (0.052)
coalition_typepublic 0.317 0.910**
(0.226) (0.358)
log(coalition_size) 0.245** 0.261** 0.255** 0.247**
(0.104) (0.110) (0.106) (0.114)
coalition_unopposedTRUE -1.623 -1.712 -1.355 -2.050
(1.563) (1.584) (1.527) (1.552)
coalition_business -0.003 -0.016
(0.013) (0.013)
presidentTrump 0.538 -0.543**
(0.376) (0.223)
coalition_typepublic × presidentTrump -1.420***
(0.485)
coalition_business × presidentTrump 0.054**
(0.027)
Num.Obs. 206 254 166 211
R2 0.051 0.035 0.122 0.083
R2 Adj. 0.032 0.019 0.088 0.056
AIC 772.3 959.6 613.1 790.0
BIC 792.3 980.8 638.0 816.8
Log.Lik. -380.144 -473.804 -298.558 -386.976
F 2.711 2.229 3.666 3.062
* p < 0.1, ** p < 0.05, *** p < 0.01

My preferred model is model 3:

# model-success-plot
m_president %>%
  tidy(conf.int = TRUE) %>% 
  filter(term != "(Intercept)") %>% 
  ggplot() + 
  geom_hline(yintercept = 0, color = "grey") + 
  aes(x = term, 
      y = estimate, 
      ymin = conf.low, 
      ymax = conf.high) + 
  geom_pointrange( )  + 
  coord_flip() +
  labs(y = "Lobbying Success", 
       x = "")
OLS Model of Coalition Lobbying Sucess with Hand-coded Data

Figure 4.1: OLS Model of Coalition Lobbying Sucess with Hand-coded Data

4.0.1.1 Modeling Congressional Support as a Mediator of Lobbying Success

To assess congressional support as a mediator in the influence of public pressure campaigns on rulemaking, I estimate the average conditional marginal effect (ACME, conditional on the number of comments from Members of Congress) and average direct effect (ADE) of mass comments using mediation analysis. Model 3 in table 4.1 replaces the dependent variable (lobbying success) with the mediator variable (the number of supportive members of Congress). Model 1 is the same as Model 1 above. Model 2 is the same but includes the proposed mediator, the number of supportive comments from members of Congress.

#TODO add headers, cut out demands, add political information box, shift principal comments over
#TODO add outcome model
knitr::include_graphics(here::here("figs", "causal-oversight-1.png"))
The Mediator Model: The Relationship Between Public Pressure and Congressional Oversight

Figure 4.2: The Mediator Model: The Relationship Between Public Pressure and Congressional Oversight

4.0.1.1.1 Mediator model (4.1):

\[ \text{Congressional support}_{ij} = \beta_0 + \beta_1 log(\text{Comments}_{ij}) + \beta_{2-n} X_{ij} + \epsilon_{ij} \tag{4.1} \]

# poisson correct model 
m_congress <- glm(coalition_congress ~  log(comments) + #comment_length +
                coalition_type +  log(coalition_size) + coalition_unopposed,
               family = "poisson",
               data = coalitions_coded %>% filter(!is.na(coalition_success))) 

# model predicting mediator
model.m <- lm(coalition_congress ~  log(comments) + #comment_length +
                coalition_type +  log(coalition_size) + coalition_unopposed,
               data = coalitions_coded %>%
                filter(!is.na(coalition_success))) 


mediator <- list("Members of Congress in Coalition (OLS)" = model.m, "Members of Congress in Coalition (Poisson)" = m_congress)

modelsummary(mediator, stars = TRUE)
Members of Congress in Coalition (OLS) Members of Congress in Coalition (Poisson)
(Intercept) -0.078* -25.472
(0.043) (2989.342)
log(comments) -0.016 -0.248
(0.011) (0.167)
coalition_typepublic 0.084* 19.180
(0.050) (2989.341)
log(coalition_size) 0.073*** 1.582***
(0.023) (0.391)
coalition_unopposedTRUE 0.078 3.169
(0.346) (42352.794)
Num.Obs. 206 206
R2 0.070
R2 Adj. 0.051
AIC 151.5 59.5
BIC 171.4 76.2
Log.Lik. -69.730 -24.761
F 3.765
* p < 0.1, ** p < 0.05, *** p < 0.01 
# FIXME include = TRUE when more obs
model.m %>%
  tidy(conf.int = TRUE) %>% 
  filter(term != "(Intercept)") %>% 
  ggplot() + 
  geom_hline(yintercept = 0, color = "grey") + 
  aes(x = term, 
      y = estimate, 
      ymin = conf.low, 
      ymax = conf.high) + 
  geom_pointrange( )  + 
  coord_flip() +
  labs(y = "Number of Legislator Comments", 
       x = "")
4.0.1.1.2 Outcome model
#TODO label "Public Pressure" and "Oversight" portions. 
knitr::include_graphics(here::here("figs", "causal-oversight-2.png"))
Integrating Public Pressure and Congressional Oversight into a Model of Lobbying in Bureaucratic Policymaking

Figure 4.3: Integrating Public Pressure and Congressional Oversight into a Model of Lobbying in Bureaucratic Policymaking

(\(y_{ij} = \text{Lobbying\ success}_{ij}\)) (4.2):

\[ y_{ij} = \beta_0 + \beta_1 log(\text{Comments)_{ij}) + \beta_2 congressional\ support_{ij} + \beta_{3-n} X_{ij} + \epsilon_{ij} \tag{4.2} \]

# model predicting DV
model.y <- lm(coalition_success ~ log(comments) + 
                coalition_congress + #comment_length + 
                coalition_type +  
                log(coalition_size) + 
                coalition_unopposed, 
              data = coalitions_coded) 

# Mediation 
med.cont <- mediate(model.m, model.y,
                    sims=1000, 
                    treat = "log(comments)", 
                    mediator = "coalition_congress")

summary(med.cont)
## 
## Causal Mediation Analysis 
## 
## Quasi-Bayesian Confidence Intervals
## 
##                Estimate 95% CI Lower 95% CI Upper p-value   
## ACME            0.00434     -0.00648         0.02   0.520   
## ADE            -0.15488     -0.26320        -0.05   0.006 **
## Total Effect   -0.15054     -0.25893        -0.04   0.010 **
## Prop. Mediated -0.01752     -0.22694         0.04   0.526   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Sample Size Used: 206 
## 
## 
## Simulations: 1000
models <- list(
  "1" = m,
  "2" = model.y,
  "3"  =  model.m
)

rows <- tibble(
  term = c("Dependent Variable"),
  `1` = c("Lobbying Success"), 
  `2` =c("Lobbying Success"), 
  `3`  = c("Members of Congress in Coalition")
)

# #broom::tidy(m_PR)
# cm = c("ej_commentTRUE" = "EJ Comment",
#        "log(comments + 1)" = "Log(Comments+1)",
#        "ej_comments_unique" = "Unique EJ Comments",
#        "ej_commentTRUE:log(comments + 1)" = "EJ Comment*Log(Comments+1)")

attr(rows, 'position') <- c(0)

# paper table
modelsummary::modelsummary( models,
                            title = "Lobbying Success and Congressional Support",
                            stars = TRUE,
                            #coef_omit = "president.*|agency.*|Intercept",
                            # coef_map = cm,
                          add_rows = rows,
                          notes = "") %>%
  row_spec(row = 1, bold = T) #%>% kable2(file = "mediation")
Table 4.1: Lobbying Success and Congressional Support
1 2 3
Dependent Variable Lobbying Success Lobbying Success Members of Congress in Coalition
(Intercept) 0.022 -0.397** -0.078*
(0.307) (0.196) (0.043)
log(comments) -0.138*** -0.157*** -0.016
(0.047) (0.052) (0.011)
coalition_typepublic -0.006 0.338 0.084*
(0.258) (0.228) (0.050)
log(coalition_size) 0.159 0.264** 0.073***
(0.121) (0.106) (0.023)
coalition_unopposedTRUE NA -1.603 0.078
() (1.564) (0.346)
coalition_congress -0.259
(0.319)
Num.Obs. 136 206 206
R2 0.067 0.054 0.070
R2 Adj. 0.046 0.031 0.051
AIC 484.1 773.6 151.5
BIC 498.7 796.9 171.4
Log.Lik. -237.051 -379.804 -69.730
F 3.160 2.297 3.765
* p < 0.1, ** p < 0.05, *** p < 0.01

Mediation analysis will require adding cases where coalitions lobbied unopposed, which we are much more likely to see in the sample of rules without mass comments.

The average effect of the logged number of comments, conditional on letters from members of congress (the ACME) is 0, with a p value of 0.52.

The average direct effect (ADE) of the logged number of comments on lobbying success is -0.15, with a p-value of 0.006.

The Total Effect of a one-unit increase in the logged number of comments is -0.15, with a p value of 0.01. -0.02 of this is mediated through mobilizing congressional attention (p-value = 0.53).

4.0.2 DV = organzation success

While it would not be appropriate to compare the lobbying success of organizations within a rulemaking (because many organizations belong to the same coalition), it may be appropriate to compare the lobbying success within the same organization across rules. This limits the analysis to organizations that lobbying on multiple policies. The key variation of interest is when organizations lobby with a large amount of public support versus when they do not.

There is still a (lesser) problem with the i.i.d. assumption here because two organizations lobbying in a coalition on one rule may mobilize each other to lobby in coalition in a different rule (in my data, lobbying coalitions are at the policy-level, since they differ from policy to policy).

orgs <- comments_coded %>% 
  filter(comment_type == "org") %>% 
  distinct(org_name, docket_id, success, Position, coalition_size, coalition_comments, president) %>% 
  count(org_name, sort = T) %>% 
  filter(n >1, !is.na(org_name)) %>%
  mutate(org_name = org_name %>% str_to_title())

orgs %>% kablebox()
org_name n
Natural Resources Defense Council 9
Sierra Club 9
Earthjustice 8
Oceana 8
Pew Charitable Trusts 7
Center For Biological Diversity 6
American Bankers Association 5
American Petroleum Institute 5
Associated Builders And Contractors 5
Association Of Oregon Counties 5
County Of Siskiyou 5
Edison Electric Institute 5
Environmental Defense Fund 5
International Association Of Drilling Contractors 5
International Bancshares Corporation 5
Materion Brush Inc.  5
National Association Of Home Builders 5
National Audubon Society 5
National Employment Law Project 5
National Mining Association 5
National Wildlife Federation 5
Nez Perce Tribal Executive Committee 5
Ocean Conservancy 5
Port Gamble S’klallam Tribe 5
Quinault Indian Nation 5
Afl-Cio 4
American Bird Conservancy 4
Blue Water Fishermen’s Association 4
Chamber Of Commerce 4
Confederated Tribes Of Warm Springs 4
Economic Progress Institute 4
Elko County 4
Farm Bureau 4
Harney County Court 4
Independent Community Bankers Of America 4
Institute For Policy Integrity 4
Kentucky Equal Justice Center 4
Montezuma County 4
National Wild Turkey Federation 4
Nature Conservancy 4
North Carolina Division Of Marine Fisheries 4
North Slope Borough 4
Northeast Seafood Coalition 4
Ocean Conservation Research 4
Offshore Operators Committee 4
Portland Cement Association 4
Public Citizen 4
Safari Club International 4
Snoqualmie Indian Tribe 4
Southern Ute Indian Tribe 4
State Of Alaska 4
Summit Lake Paiute Tribe 4
U.s. Chamber Of Commerce 4
Union Of Concerned Scientists 4
Aca International 3
Aircraft Owners And Pilots Association 3
American Staffing Association 3
Arizona Game & Fish Department 3
Associated General Contractors Of America 3
Association To Preserve Cape Cod 3
Baker County 3
Better Markets 3
Catholic Charities Usa 3
Center For Regulatory Effectiveness 3
Chesapeake Bay Foundation 3
Citizens’ Alliance For Property Rights 3
Columbia River Inter-Tribal Fish Commission 3
Competitive Enterprise Institute 3
Confederated Tribes Of Grand Ronde 3
Conference Of State Bank Supervisors 3
Consumer Bankers Association 3
Consumer Reports 3
Domestic Energy Producers Alliance 3
Duchesne County 3
Endangered Species Coalition 3
Energy And Wildlife Action Coalition 3
Faith For Just Lending 3
Florida Fish And Wildlife Conservation Commission 3
Florida Wildlife Federation 3
Friends Of The Clearwater 3
Governor’s Office 3
Harding County 3
Hr Policy Association 3
Maine Coast Fishermen’s Association 3
Maryland Consumer Rights Coalition 3
Mcintyre & Lemon, Pllc 3
Mesa County 3
Minnesota Deaprtment Of Natural Resources 3
Naacp 3
National Association Of Conservation Districts 3
National Association Of Counties 3
National Association Of Manufacturers 3
National Congress Of American Indians 3
National Employment Lawyers Association 3
National Endangered Species Act Reform Coalition 3
National Parks Conservation Association 3
National Women’s Law Center 3
Network Lobby For Catholic Social Justice 3
North Dakota Game & Fish 3
Nyc Department Of Consumer Affairs 3

384 organizations lobbied on more than one rule in the hand-coded data, some on as many as 9 rulemaking dockets. This yields a total of 986 observations of an organization lobbying on a docket that also lobbied on some other docket. (Note: this is a undercount due to imperfect standardization of organization names).

At the organization level, the appropriate analysis is a difference-in-difference design. We know the success of each organization when it does and does not participate in a lobbying coalition that mobilizes public pressure (at least each organization that I can use for this analysis). The difference within an organization is now the key variation.

\(Y_{ij} = \boldsymbol{\beta}_1 \textbf{Comments}_{ij} + \gamma_{i} + {\beta}_2\text{Coalition Size}_{ij} + {\beta}_3\text{Support}_{ij} + {\beta}_4\text{President}_{j} + \epsilon_{ij}\)

Where \(Y_{it}\) represents the level of success that organization \(i\). \(\gamma_{ij}\) is a fixed effect for the organization. This fixed effect accounts for the organization’s characteristics. This difference-in-difference design ensures that coefficient \(\boldsymbol{\beta}_1\) captures variation related to changes in levels of public pressure, not other factors that may vary across organizations.

\({\beta}_2\) captures the effect of coalition size on lobbying success of organization \(i\) on rule \(j\). \({\beta}_3\) captures the difference in the success of organization \(i\) when they support proposed policy \(j\) rather than oppose it. \({President}_{j}\) is a dummy for whether policy \(j\) was proposed by President Trump rather then-president Obama’s administration.

Assuming that organizations have parallel trends in their level of success given a level of support, \(\boldsymbol{\beta}\) represents the average effect of changing levels of public pressure on an organization’s lobbying success.

Estimates in the table below show the results of this model. It suggests that the same organization was less effective when it mobilized more comments, more successful when they supported the rule, and less successful under president Trump than President Obama.

The negative correlation between lobbying success and the number of mass comments is likely due to campaigns “going down fighting”–not trying to influence policy. The fact that organizations are more likely to get the outcome they seek when they already support the rule makes sense because the agency is more likely to be sympathetic to their requests. The fact that the average organization was less likely to see its desired policy changes under President Trump is likely due to asymmetry in mobilizing organizations, with more organizations on the left than the right in this sample of rules. (Note: this may change in the broader sample.)

library(fixest)

m_fe_org = feols(success ~ log(coalition_comments) + coalition_size + Position + president | org_name,
               data = comments_coded %>% 
  filter(comment_type == "org") %>% 
  distinct(org_name, docket_id, success, Position, coalition_size, coalition_comments, president) %>% filter(president != "Bush"))

models <- list("Lobbying Success" = m_fe_org)

modelsummary(models, stars = T)
Lobbying Success
log(coalition_comments) -0.097**
(0.044)
coalition_size 0.001
(0.003)
PositionSupports rule 0.597**
(0.264)
presidentTrump -0.551**
(0.232)
Num.Obs. 1892
R2 0.915
R2 Adj. 0.412
R2 Within 0.094
R2 Pseudo
AIC 6204.1
BIC 15182.1
Log.Lik. -1483.036
FE: org_name X
Std. errors Clustered (org_name)
* p < 0.1, ** p < 0.05, *** p < 0.01
Casas, Andreu, Matthew J Denny, and John Wilkerson. 2017. “Legislative Hitchhikers: Re-envisioning Legislative Productivity and Bill Sponsorship Success.”
Judge-Lord, Devin. 2017. “Measuring Change and Influence in Budget Texts.” In. American Political Science Association Annual Conference.
“Lost in the Flood?: Agency Responsiveness to Mass Comment Campaigns in Administrative Rulemaking.” 2020. Regulation & Governance.
Yackee, Jason Webb, and Susan Webb Yackee. 2006. “A Bias Towards Business? Assessing Interest Group Influence on the U.S. Bureaucracy.” Journal of Politics 68 (1): 128–39. https://doi.org/10.1111/j.1468-2508.2006.00375.x.

  1. Where a new presidential administration used the same docket number to solicit comments on a proposed rule that a previous administration used, I count these as separate rulemaking dockets. I do so because the second policy is usually reversing or going in the opposite direction as the previous administration’s policy solicited comments. The same organizations often comment but with the opposition positions. Support becomes opposition and vice versa.↩︎

  2. For more about n-gram window functions and comparisons with related partial matching methods such as the Smith-Waterman algorithm, see Casas, Denny, and Wilkerson (2017) and Judge-Lord (2017).↩︎